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Abstract 

This paper demonstrates how alumni survey research can be designed to focus assessment 
of master's degree programs on student outcomes and be responsive to program goals, policy 
concerns of administrators, instructional values of the faculty, and standards of professional 
practice. Based on a completed study of 541 alumni/ae of master's degree teacher education 
programs at a large northeastern university, this paper presents critical design issues, analysis 
principles and selected findings relevant to assessment of other professional master's degree 
programs. Results confirm the impact of satisfaction with courses, perception of intellectual 
challenge and the quality of professional preparation on graduates' overall evaluation of the 
program and on their reevaluation of their choice of this particular program. 
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Introduction 

This paper presents critical design and analysis principles relevant to conducting graduate 
follow-up studies for the assessment of professional master's degree programs. Based on a 
completed assessment study of a master's degree teacher education program at a large 
northeastern university, the paper demonstrates how a well designed instrument can address a 
broad range of assessment issues including policy concerns of administrators, instructional 
values of the faculty, standards of professional practice, and particular goals of a professional 
master's level educational program. To inform the design of future assessment studies, the paper 
presents a conceptual framework, a research design plan, identification of relevant issues and 
appropriate analytical techniques. 

Review of the Literature 

Graduate Level Assessment 
A review of the literature indicates that graduate level assessment ought to be given 
greater attention in the higher education assessment movement. In their review, Kaylor and 
Johnson (1994) observe that graduate professional education is seldom addressed in discussions 
of outcomes assessment. The authors suggest further that some accommodation needs to be 
made in order to integrate graduate professional education into the current assessment 
movement. Essential to the integration of assessment in professional graduate education is an 
awareness that professional education is inherently holistic, involving the synthesis of technical 
knowledge, skills and judgments (Elman & Lynton, 1986). An understanding that the actual 
practice of the profession must be responsive to a wide array of social, economic, and political 
factors is also relevant to effective assessment of graduate education. "It is this necessary 
responsiveness in a wider system of practice that the institutional researcher can use in the 
creation of a more substantive assessment program" (Kaylor & Johnson, 1994, p. 12). 

Assessment literature highlights in particular the need for increased attention to master's 
level education. Historical analysis has revealed that assessment of master's degree programs in 
the United States was rarely mentioned in the literature until the 1970's. Further, once 
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undertaken, master's le^el assessments were identified as a priority concern in graduate education 
(Pelczar & Frances, 1984). A review of the initial assessment efforts revealed two liniitations: a 
limited focus on program resources and a lack of attention to student outcomes (Conrad & Egan, 
1990). In response to these limitations, critics have recommended expanding the criteria to 
ensure a comprehensive basis for assessment. Conrad and Egan (1990) identify six criteria: 
faculty, students, resources, learning environment, curriculum and placement of alumni. More 
recently, authors of A Silent Success (Conrad, Haworth & Millar, 1993) identify attributes of 
high quality master s experiences which could form the basis for a comprehensive assessment 
program. Also, for some time, researchers have advocated that master's degree assessment 
efforts give greater attention to the students who are the primary 'stakeholders' in this enterprise 
(e.g., Clark, 1979; Kirkwood, 1985; & Conrad & Egan, 1990). 

Alumni Research in Assessment 

This paper demonstrates how survey research can be used to focus assessment of 
master's degree programs on student outcomes while including evaluation of the faculty, 
curricula and the learning environment. The relevance of alumni research to administrative 
decision making and various university functions has been recognized for some time (Pace, 
1979). Recently, alumni research has assumed increasing importance for its potential 
contribution to outcomes assessment (Pike, 1990). Alumni surveys provide a basis for evaluating 
how well the knowledge and skills developed through the educational program relate to the 
knowledge and skills required in the workplace (Jennings, 1989). As Williford and Moden 
(1989) observe, a unique feature of alunini surveys, compared with surveys of enrolled students, 
is the capability of documenting students' assessment of the quality of their educational 
experience tempered by their experiences since graduation. Alunini potentially offer an 
objective perspective given their distance from involvement with the program (Khalil, 1990). 

Results from alumni research also provide a basis for examining the relationship between 
satisfaction with college and academic major (Richardson, 1993), perception of learning during 
college (Pike, 1993b), and subsequent work experience (Pike, 1993a). Of particular importance 
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to institutional researchers is the realization that in order to serve any of these functions 
effectively, the alumni research study needs to be designed and the information presented in the 
appropriate form and at the appropriate level of detail for the intended audiences and users of the 
results (Moden and Williford, 1988). 

Design Issues 

Identification of the Population 

The identification and size of the survey population are critical design issues that affect 
the ability to analyze variations in the data and to make inferences to populations of interest. 
Generally it is advisable to determine, prior to conducting the study, what subgroup analyses will 
be conducted, and to estimate not only the overall response rate but also the response rates for 
individual departments or classes for which inferences might be made. Statistical formulae, 
taking into account levels of statistical significance and power, may also be used to determine 
sufficient sample sizes to support tests of significance and inferences to the intended population. 

The population for the present study included 580 alumnl^ae who graduated from 1987 
through 1993 with an M.Ed, degree from the School of Education in one of the following teacher 
education programs: Elementary Education, Secondary Education, Special Education, Early 
Childhood Education and Reading. Some fifty full-time faculty are engaged in these 
professional education programs which enroll approximately 200 full-time and 400 part-time 
graduate students and 550 full-time undergraduate students. The School of Education is one of 
several professional schools in a large, private university with 591 full-time faculty, 4300 
graduate and professional students and a total enrollment of 14,500. 

Data were obtained by means of a mailed survey sent during the 1994 spring semester. 
Follow-up data collection procedures included cover letters from the Dean, postage paid return 
envelopes with each survey mailing, a reminder post-card to the total population after the initial 
mailing, and two complete follow-up survey packages to all non-respondents. These follow-up 
procedures yielded a response rate of 64 percent; of the 541 deliverable surveys, 347 were 
returned. 
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Design of the Survey Instrument 
The survey included three major sections: A. Evaluation of the Masters Program, B. 
Employment Experience Prior and Subsequent to the Master's Program, and C. A Profile of 
Survey Respondents. The evaluation section, the core segment of the survey, included 
questions eliciting graduates' expectations from and satisfaction with various components of the 
program as well as their assessment of how well the program prepared them for various aspects 
of their professional practice. The employment experience segment focused on graduates' 
employment seeking experience including their degree of difficulty in finding employment and 
the types of positions sought; a history of their teaching experience including the types of 
positions held, grade levels, job settings and types of institutions in which they taught; current 
employment status; level of job satisfaction and professional challenges encountered in teaching. 
The profile of survey respondents elicited information on respondents' age, gender, 
racial/ethnic background, income and undergraduate education; their reasons for choosing and 
sources of financing the Master's Program; and the nature and extent of education pursued and 
planned following the completion of the Master's Program. The following discussion identifies 
the substantive issues addressed in the evaluation section and in the segment of the employment 
section dealing with professional challenges. These two areas were most pertinent to the 
assessment of the program's effectiveness. 

Evaluation Section 

This survey illustrates the importance of conceptualizing student outcome from the 
perspective of various constituencies - graduates, faculty and professional accrediting 
organizations. These perspectives were identified through interviews, discussions and analysis 
of institutional and professional literature. 

Graduates' Perspective . In a structured questionnaire format, survey respondents were 
presented with the following set of expectations and asked to indicate which ones reflected their 
initial expectations of the master's degree teacher education program and whether or not their 
expectations were achieved: 
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• To further one's own intellectual development; 

• To achieve the next step in pursuit of a doctorate; 

• To advance professionally; 

• To obtain initial teacher education training; 

• To obtain necessary credentials for one's career; 

• To become more competitive in the job market; and 

• To increase one's salary level. 

These expectations were intended to reflect the mu.ltiple goals of a graduate professional degree 
program, i.e., intellectual growth, professional development and career preparation. Respondents 
were also provided with the option of identifying their own additional expectations. Policy and 
analysis considerations suggest that student expectations and the fulfillment of these expectations 
should be integral components of teacher education follow-up studies. From a policy perspective, 
teacher preparation programs should be accountable to the students they educate as well as to the 
standards of the profession. Further, analyses of these data, showing the relationship between 
students' expectations and the reality of their educational experience, may yield valuable insights 
relevant to explaining variation in graduates' satisfaction with their educational program. 

Two survey questions focused on graduates* satisfaction with the program; both 
questions used four point Likert response scales. These questions were designed in collaboration 
with the faculty who recommended that graduates' satisfaction with their courses be measured at 
the department level rather than at the individual course level. The first, question focused on the 
following general aspects of the master's program: class lectures/professors' presentations, class 
discussions/exchanges with peers, assigned readings, course assignments/projects, balance 
between theory and practice, coverage of current issues, relevance to the real world, use of 
technology, and professor's methods of evaluating students. The second question addressed the 
following specific components of the teacher education program: School of Education courses in 
general, courses in the student's own major, "methods" courses (e.g. Science), "theory" courses 
(e.g. Modem Educational Thought), foundations courses (e.g. Educational Measurement), 
Psychology - Human Development courses. Arts and Sciences courses (not in education), work 
with faculty on research projects, practicum experience, responsiveness of the Field Office, 
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academic and career advising from faculty, Educational Resource Center, university library 
holdings in education, and intellectual environment. 

Graduates were also asked to rate, on a four point Likert scale, the level of intellectual 
challenge offered by the following aspects of the master's program: class lectures/professors' 
presentations, class discussions/exchanges with peers, assigned readings, and course 
assignments/projects. 

Faculty Perspective . The faculty's perspective on student outcomes, presented in a paper 
entitled 'Andover Themes', provided the source for another question in the survey. The themes 
emanated from discussions at academic retreats held at the Andover Inn in Massachusetts. This 
document identified the follov/ing themes reflecting the faculty's philosophy and goals for the 
teacher prepai*ation programs: learners as constructors of meaning; reflection on theory, 
research, and practice; feedback from practice informing theory and research; diverse models and 
methods for diverse learners; and promoting social justice. Two questions were designed to 
address these themes. First, on a five point Likert scale, respondents were asked to rate how well 
their master's program enabled them to achieve the following objectives: 

♦ Possess an adequate knowledge base to develop and evaluate your own philosophy; 

♦ Reflect on and evaluate their own practice; 

♦ Translate the theories learned into practical knowledge and teaching strategies; 

♦ Handle uncertainty by constructively seeking the best solution to problems; 

♦ Deal effectively with complexity resulting from multiple teacher roles; and 

♦ Design and execute classroom-based research. 

In another question, graduates were asked how well the program helped them develop the 
capability to cope with students: 

♦ With different ability levels in the same class; 

♦ From different socioeconomic backgrounds; 

♦ From diverse racial/ethnic, cultural backgrounds; and 

♦ In an inner-city school system. 

In summary, these two questions elicited graduates' perception regarding how well the program 
prepared them to engage in reflective practice and to cope effectively with diversity in the 
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classroom - both major themes in the faculty's statement of goals for the teacher education 
programs. 

Professional Accreditation Standards . The external accrediting board's standards of 
professional practice provide yet another source for identifying relevant student outcomes. In 
this survey, the 'Common Knovv'ledge Standards' articulated by the Commonwealth of 
Massachusetts for accreditation of teacher education programs were used as a basis for 
generating a question focused on student outcomes in terms of standards of professional practice. 
These standards address several dimensions of teaching, including Communication, Instructional 
Practice, Evaluation, Problem Solving, Equity and Professional Responsibility. In a structured 
question format, respondtats were asked to rate, on a five point Likert scale, hqw well their 
master's program enabled them to achieve the following objectives related to these standai'ds: 

♦ Communicate the goals of learning clearly to students; 

♦ Integrate knowledge of subject, method, and individual differences; 

♦ Design and use evaluative procedures appropriate for students in different groups; 

♦ Use inquiry methods to create an effective learning environment; 

♦ Use relevant support systems within and outside the school system; and 

♦ Show respect for the unique students' developmental and cultural needs. 

Employment Section ■ Professional Challenges 

For the purpose of assessing how well the master's program prepared graduates for their 
professional practice, an effort was made to identify the professional challenges alumni/ae 
encountered in their post-graduate teaching experience. Respondents were presented with a list 
of potential work related challenges and asked to indicate how challenging each of these issues 
was for them during their teaching assignments after earning their master's degree: collaborate 
with others, develop rapport with students, develop students' critical thinking skills, evaluate 
students' performance, evaluate their own teaching, maintain discipline, motivate students, plan 
cb^s instruction, relate to administrators, relate to parents, train students in problem-solving 
methods, and work with mainstreamed students. 
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Analysis Techniques 

Development of Scales 

Procedures employed in the analysis of this master's alumni survey illustrate how 
statistical techniques can be used tc^ simplify survey data and to create reliable measures of the 
program dimensions being evaluated. Initially, factor analysis was employed with an extensive 
number of questionnaire items related to graduates' evaluation of the program and their report of 
challenges encountered in their professional practice. Results from factor analyses indicated 
which individual items were correlated with each other and what underlying dimensions were 
represented in the data. Scales were then created by combining similar items into one measure; 
items with high factor loading or weights on a particular factor w«^re chosen to be included in the 
scale. Prior to using the scales in the analysis, alpha reliability coefficients were computed to 
determine the internal consistency of the scales. 

Reliability analysis confirmed the internal consistency of eight scales. For purposes of 
analysis and discussion, these eight scales are classified into two broad categories: Program 
Evaluation and Professional Experience scales. There are six Program Evaluation scales and two 
Professional Experience scales. Given the nature of the issues addressed, a modified version of 
these scales may be relevant to the evaluation of other programs. The content, while focused on 
teacher education, could be easily adapted to other professional degree programs. 

Program Evaluation Scales 

The six Program Evaluation scales represent different dimensions of graduates' 
evaluation of their education: their satisfaction with various program components, their 
perception of how well the program prepared them for their professional practice, and their 
assessment of the level of intellectual challenge offered in the program. To reflect these different 
dimensions, the program evaluation scales are presented in Table 1 as Program Satisfaction 
scales. Assessment of Professional Preparation scales, and Assessment of the Program's 
Intellectual Challenge scale. As shown in Table I, there are two Program Satisfaction scales - 
Satisfaction with Courses and Satisfaction with Practical Relevance; three Assessment of 
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Professional Preparation scales - Standards of Professional Practice, Preparation for Reflective 
Practice, and Prep?^ration for Diversity; and one Assessment of the Program's Intellectual 
Challenge scale. Table 1 presents the statistical properties for each scale including the mean and 
standard deviation for the total group on these scales and the alpha reliability coefficient 
measuring the scales' internal consistency. As shown, the reliability coefficients are quite high, 
ranging from .80 to .92. 

Table 1 

Statistical Properties of the Program Evaluation Scales 

No. of Range of 

Scales Mean S.D. Reliabihty Items Responses 

Program Satisfaction Scales Low - Hig h 

Satisfaction with Courses 

Satisfaction with Practical Relevance 3.22 . 1 1 

Assessment of Professional 
Preparation Scales 

Standards of Professional Practice 

Preparation for Reflective Practice 

Preparation for Diversity 

Assessment of the Program's 
Intellectual Challenge Scale 

Program Satisfaction Scales . Items included in the first scale, Satisfaction with 
Courses are based on responses to questions asking alumni to. rate their satisfaction with courses 
or aspects of courses in the master s program, including class lectures/professors' presentations, 
class discussions. School of Education courses in general, and courses in the student's own 
major. The second scale. Satisfaction with the Program's Practical Relevance measures a 
crucial aspect for teacher education programs today, i.e., the program's perceived success in 
educating student teachers regarding real world issues and in preparing them to apply theory in 
the world of practice. 



3.31 


.11 


.88 


10 


1-4 


3.22 


.11 


.80 


3 


1 -4 


4.02 


.20 


.86 


6 


1-5 


3.90 


.29 


.90 


6 


1 -5 


3.25 


.17 


.92 


3 


1-5 


3.07 


.11 


.83 


4 


1 -4 
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Assessment of Professional Preparation Scales. One of the benefits of an alumni survey 
is the opportunity it provides to elicit graduates' evaluation of their education through the lens of 
their experience. Relevant to this evaluation is the perceived quality of the preparation provided 
for professional practice. As noted previously, this alumni survey included questions designed 
specifically to obtain graduates' evaluation of the professional preparation they received in 
relation to certain criteria - professional standards, faculty goals, and the School of Education's 
commitment to diversity. 

The first scale in this category. Standards of Professional Practice, contains six items 
reflecting selected professional accreditation standards, such as communicating the goals of 
learning clearly, understanding and showing respect for individual differences among students, 
^nd using relevant support systems. The second scale. Preparation for Reflective Practice, 
contains six items reflecting selected themes from the faculty's philosophy of teaching including 
enabling students to develop a knowledge base, make the knowledge their own, apply the 
knowledge in uncertain and complex environments, and enhance the knowledge base of the 
profession by designing and conducting research based on one's own experience. The third 
scale. Preparation for Diversity, focuses primarily on graduates' assessment of how well their 
master's program helped them develop the capability to cope with various aspects of diversity in 
their teaching of students from diverse socioeconomic, racial, ethnic and cultural backgrounds 
and specifically those in an urban setting. 

Pro gram's Intellectual Challenge Scales^ One question in the alumni survey focused 
specifically on graduates' assessment of how intellectually challenging they found the following 
aspects of their master's program: class lectures/professors' presentations, class 
discussions/exchanges with peers, assigned readings, and course assignments/projects. Factor 
analysis established one dimension underlying responses to these questions. These questions 
were, therefore, combined into one scale. Assessment of the Program's Intellectual Challenge, 
with a high reliability of .83. 
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Professional Experience Scales 

The two scales in this category are based on one question in the survey which presented 
respondents with a list of work related challenges, often mentioned by teachers, and asked them 
to indicate the degree to which these issues presented a challenge to them in their teaching 
assignments after earning their master's degree. The first professional experience scale, 
Teachers* Relationship with Students represents the various roles a teacher may assume with 
respect to students including teacher, advisor, mentor and disciplinarian. The second scale. 
Teachers* Other Professional Roles refers to the teachers' relationship with others and to his or 
her ability to evaluate his or her own teaching. The statistical properties for these scales are 
presented in Table 2. As shown, the reliability coefficients are quite strong, .87, for the first 
scale concerning challenges in teachers' relationships with students, and moderately strong, .76, 
for the second scale concerning challenges in teachers' other professional roles. 



Table 2 

Statistical Properties of the Professional Experience Scales 



Professional Experience Scales 


Mean 


S.D. 


Reliability 


No. of 
Items 


Range of 
Responses 


Professional Challenges Encountered in 
Teachers' Relationship with Students 


2.44 


.35 


.87 


7 


Low - High 
1-5 


Professional Challenges Encountered in 
Teachers' Other Professional Roles 


2.29 


.23 


.76 


4 


1-5 



Master s Assessment 14 

Selected Results 

Relationship between Evaluation of Program Dimensions and Overall Evaluation 
Results from correlation analyses with computed scales document a relationship between 
the evaluation of certain program dimensions and the overall evaluation. Statistically significant 
relationships were found between the six program evaluation scales and graduates' overall 
evaluation of how well the master's program prepared them for a teaching position. As shown in 
Table 3, the strongest relationship was found with graduates' perception of the preparation 
received for Reflective Practice (r = .73 p < .01) followed by Standards of Professional Practice 
(.61 p < .01), Satisfaction with Courses (r= .57 p < .01), Satisfaction with the Program's Practical 
Relevance (r = .54 p < .01), Assessment of the Program's Intellectual Challenge (r =.47 p < .01), 
and Preparation for Diversity (r =.45 p < .01). 

Table 3 

Correlations between Program Evaluation Scales and Graduates' Overall Evaluation of the 

Master's Teacher Education Program 



Program Evaluation Scales Correlation 



Program Satisfaction Measures 

Satisfaction with Courses .57** 

Satisfaction with the Program's Practical Relevance .54** 

Assessment of Professional Preparation Scales 

Standards of Professional Practice .61** 

Preparation for Reflective Practice .73** 

Preparation for Diversity .45** 

Assessment of the Programs' Intellectual Challenge .47** 

** p< .01 



Of particular interest in these results is the strong relationship between perceived 
preparation for the profession and overall evaluation of the program. The high correlations for 
the two Assessment of Professional Preparation scales - Standards of Professional Practice and 
Preparation for Reflective Practice - indicate that graduates' perception of how well the program 
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prepared them to meet the standards and responsibilities of their profession strongly influences 
their evaluation of their educational preparation. 

Assessment of Intellectual Challenge and Overall Evaluation 
Results from Chi square and correlation analyses indicate that there is a significant 
positive relationship between graduates' ratings of the intellectual challenge of various aspects of 
their program and their ratings of how well the program prepared them for a teaching position. 
Analysis based on individual questionnaire items revealed statistically significant positive 
relationships between graduates' overall rating of the professional preparation received through 
the master's program and the perceived intellectual challenge of each of the following: class 
lectures (r =.40 p < .01), assigned reaJings ( r =.42 p < .01), and course assignments and projects 
(r=.41 p<.01). 

Data presented in Table 4 illustrates the moderately strong, statistically significant 
relationship found between graduates' assessment of the intellectual challenge of class lectures 
and their overall evaluation of the master's program. As shown, the majority of those who found 
the class lectures to be very challenging, 52.1 percent, also reported that the program prepared 
them 'Very Well' for their professional practice. In contrast, the majority of those who found the 
class lectures only 'Slightly or Not At All' challenging, 57.7 percent, also rated the program from 
'Very Poorly to Fair'. 

Table 4 

Assessment of Intellectual Challenge of Class Lectures and Overall Evaluation of the Program 

Overall Evaluation 



Intellectual Challenge of Very poorly 
Class Lectures to Fair Well Very Well Total 

Not at all/Slightly 57.7 % (30) 36.5 % (19) 5.8 % (3) 100.0 % (52) 

Moderately 26.2 (49) 51.9 (97) 21.9 (41) 100.0 (187) 

Very 9.4 (9) 38.5 (37) 52. 1 (50) 100.0 (96) 

X2 = 65.03 p<.001 



Master's Assessment 16 

Predicting Overall Evaluation of the Program 

Multiple regression results, presented in Table 5, show that controlling for employment 

satisfaction, satisfaction with the program's preparation for reflective practice, satisfaction with 

courses, and perceived level of intellectual challenge offered by the program significantly predict 

overall evaluation of the program. Employment satisfaction, entered first in the equation as a 

covariate, explains five percent of the variance; satisfaction with the program's preparation for 

reflective practice, course satisfaction and perceived level of intellectual challenge explain an 

additional forty-seven, five and one percent respectively of the total variance in overall 

satisfaction. The F ratio for the total equation is 96.79 (p < .001). 

Table 5 
Regression Results: 

Predicting Overall Evaluation of the Program From Course Satisfaction and Assessment of the 
Intellectual Challenge and Preparation Offered by the Program 



Order of 
Entry 


Predictor 


Bivariate 
r 


Multip!- 
R 


R 

Square 


R2 
Change 


Beta 
Coefficient 


F 

Ratio 


1 


Covariate: 

Employment Satisfaction 


.22 


.22 


.05 


.05 


.09 


5.7* 


2 


Preparation for 
Reflective Practice 


.72 


.72 


.52 


.47 


.50 


g2 7*** 


3 


Satisfaction with Courses 


.62 


.75 


.57 


.05 


.23 




4 


Perceived 

Intellectual Challenge 


.49 


.76 


.58 


.01 


.13 


7.5** 



The F ratio for the equation is 96.79 
* p < .05 
** p<.01 
*** p<.001 



Predicting Graduates' Reevaluation of their Program Choice 
Discriminant analysis was performed to predict graduates' responses to the question, "If 
you were to make the decision over again regarding this Master's Program, which course of 
action would you follow?" Respondents were classified in two groups, those who would make 
the same choice and those who would make a different choice. Six variables were included as 
predictors: five Program Evaluation scales - Satisfaction with Courses, Standards of Professional 
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Practice, Preparation for Reflective Practice, Preparation for Diversity, Assessment of the 
Program's Intellectual Challenge and Employment Satisfaction. Results are presented in Table 6. 

Table 6 

Discriminant Analysis Results: 
Predicting Graduates' Reevaluation of the Program Choice 





Structure 


Percent 


Predictors 


Coefficients 


Correctly Classified 


Satisfaction with Courses 


.85 


81.93 


Preparation for Reflective Practice 


.66 




Standards of Professional Practice 


.63 




Assessment of Intellectual Challenge 


.49 




Employment Satisfaction 


.45 




Preparation for Diversity 


.39 




Canonical Correlation 


.51 


. X2 = 75.32 df=6 p<.001 



As shown in Table 6, the magnitude of the structure coefficients indicate that three 
Program Evaluation scales are highly correlated with this function - Satisfaction with Courses, 
Preparation for Reflective Practice and Standards of Professional Practice. Although to a lesser 
degree, Assessment of the Program's Intellectual Challenge, Employment Satisfaction, and 
perceived Preparation for Diversity are also positively correlated with this function. The 
canonical correlation of .51 indicates that this function explains 26 percent of the variance in 
graduates' reevaluation of their program choice. This discriminant function accurately predicts 
the choice of 82 percent of the respondents. 

Discussion 

This paper illustrates how alumni survey research can be used effectively in assessment 
studies, particularly of professional graduate level programs. The discussion of the design of the 
instrument illustrates how to address policy concerns of administrators, instructional values of 
the faculty, standards of professional practice and particular goals of a professional master's level 
educational program. 
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This study responds to the recommendation that master's degree assessment efforts give 
greater attention to the students who are the primary 'stakeholders' in this enterprise (e.g., Clark, 
1979; Kirkwood, 1985; & Conrad & Egan, 1990), and that there be a comprehensive basis for 
assessment including faculty, students, resources, learning environment, curriculum and 
placement of alumni (Conrad and Egan, 1990). By addressing challenges encountered in 
professional practice and graduates' assessment of how the program might be improved to meet 
these challenges, the survey illustrates a unique value of alumni research. Alumni surveys 
provide a basis for evaluating how well the knowledge and skills developed through the 
educational program relate to the knowledge and skills required in the workplace 
(Jennings, 1989). 

While recognizing the need to involve various constituencies in planning an alumni 
survey, the paper also establishes a unique role for the researcher, i.e., to address critical design 
issues such as the identification of the population and selection of the sample; the design of the 
survey instrument; and the selection of appropriate analytical techniques. The discussion of 
design issues emphasizes the importance of planning future analyses during the initial planning 
stages and of conceptualizing student outcomes from the perspective of various constituencies. 
Analysis techniques presented in the paper illustrate the use of bivariate and multivariate 
procedures in creating reliable measures and addressing important assessment questions. 

Results from these analyses confirm the impact of satisfaction with courses, perception 
of intellectual challenge and the quality of professional preparation on graduates' overall 
evaluation of the program and on their reevaluation of their choice of this particular program. 
Both the design and findings from this research are potentially relevant to the design of future 
assessment studies of professional master's degree programs. 
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